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PEER  RATING  RESEARCH:  ANNOTATED  BIBLIOGRAPHY 


INTRODUCTION 

With  the  current  limitations  on  military  spending  in  the  U.S.  Air 
Force,  assigning  high  potential  personnel  to  training  programs 
and/or  to  jobs  is  of  especially  serious  concern.  Clearly,  the  costs 
of  selecting  or  training  individuals  who  prematurely  terminate 
employment  or  training  in  the  Air  Force  as  a result  of  poor  or 
non-adaptive  performance  represents  a critical  loss. 

One  way  to  reduce  this  kind  of  loss  is  through  early  identifi- 
cation of  high  risk  individuals  likely  to  fail  in  training  or  on  the 
job,  and  the  Air  Force  has  employed  various  approaches  to  identify 
such  high  risk  personnel.  Educational  data,  and  aptitude,  biographi- 
cal, and  vocational  interest  information  have  all  been  used  in  efforts 
to  improve  selection  and  placement  decisions. 

Another  indication  of  potential  for  later  success  in  the  Air 
Force  has  been  provided  by  peer  ratings.  During  basic  military 
training,  peer  ratings  have  been  used  to  select  personnel  for  assign- 
ment to  selected  career  fields  under  the  Human  Reliability  Program. 

The  purpose  of  the  research  to  be  conducted  in  the  present  program 
is  to  explore  the  utility  of  peer  ratings  for  predicting  success  in 
other  Air  Force  settings.  In  particular,  the  present  project  is 
evaluating  the  ability  of  peer  ratings  to  predict  success  in  basic 
training.  Future  efforts  within  the  program  will  assess  relationships 
between  these  peer  ratings  generated  by  basic  trainees  and  a)  success 
in  technical  training;  and  b)  ef feet i veness  in  subsequent  job 
performa nee. 

As  a first  step  in  the  research  program,  personnel,  industrial,  and 
social  psychology  journals  and  technical  reports  were  reviewed  to 
identify  studies  employing  peer  ratings.  This  report  contains  an 
annotated  bibliography  describing  all  studies  found  in  this  literature 
review.  Knowledge  of  previous  peer  rating  research,  most  notably  the 
procedures  used  for  scoring  peer  evaluations  and  statistical  methods 
employed  for  assessing  the  reliability  and  validity  of  such  ratings, 
should  substantially  aid  efforts  to  evaluate  the  usefulness  of  peer 
ratings  for  predicting  success  in  the  U.  S.  Air  Force. 


1.  Alt?'  .ni,  L.  M.  , 6 Yimer,  M.  An  investigation  of  the 
.o'  onship  between  colleague  rating,  student  rating, 

> -search  productivity,  and  academic  rank  in  rating 
instructional  effectiveness.  Journal  of  Educational 
Psychology , 1973,  6^,  27^-277 • 

Neither  peer  nor  student  ratings  of  college 
instructors  were  found  to  be  related  to  research 
productivity.  Peer  ratings  correlated  highly  with 
academic  rank,  suggesting  that  reputation  influenced 
the  ratings. 

2.  Alfonso,  R.  J.  Will  peer  supervision  work?  Educat iona 1 

Leadership,  1977,  59A-601. 

This  report  discusses  the  advantages  and  disadvantages 
of  peer  evaluation  in  teaching.  The  author  concludes  that 
peer  evaluation  is  useful,  but  should  be  combined  with 
supervisory  evaluation. 

3.  Amatora,  Sister  M.  Pupil  evaluation  or  teacher  evaluation 
in  personality?  Progressive  Education,  1953.  3_^,  AA-A5. 

Peer  and  teacher  ratings  of  children's  personality 
traits  showed  little  agreement,  though  the  interrater 
agreement  within  groups  was  high. 

A.  Amatora,  Sister  M.  Contrasts  in  boys'  and  girls'  judgments 
in  personality.  Child  Deve I opmen t , 195A,  2j>,  51-62. 

Boys  and  girls  rated  peers  of  their  own  sex  higher  on 
desirable  personality  traits  than  they  did  peers  of  the 
opposite  sex. 

5.  Amatora,  Sister  M.  Validity  in  self  evaluation.  Educa- 
tional and  Psychological  Measurement,  1956,  H^,  I 19* 126. 

Self  and  peer  ratings  of  children  on  The  Child 
Personality  Scale  showed  moderately  high  correlations  for 
the  majority  of  the  scales. 


6.  Amir,  Y.,  Kovarsky,  Y.,  & Sharon,  S.  Peer  nominations  as 
a predictor  of  multistage  promotions  in  a ramified 
organization.  Journal  of  Applied  Psychology,  1970,  5*t, 

462-469-  ~~ 

Peer  nominations  of  officer  potential  and  military 
performance  of  Israeli  soldiers  successfully  predicted 
subsequent  promotions.  This  predictive  relationship  held 
over  a long  period  of  time  and  across  different,  widely 
divergent  groups.  Also,  the  peer  nominations  predicted 
performance  better  than  objective  test  scores. 

7.  Arbuckle,  D.  S.  Client  perception  of  counselor  personality. 
Journal  of  Counseling  Psychology,  1956,  .3,  93 ~ 96 - 

Peer  ratings  were  used  to  differentiate  high  and  low 
competence  groups  of  counseling  trainees.  The  two  groups 
were  found  to  differ  significantly  in  terms  of  scores  on 
several  self-report  personality  scales. 

8.  Astington,  E.  Personality  assessments  and  academic  per- 
formance in  a boys'  grammar  school.  British  Journa 1 of 
Educational  Psychology,  I960,  30_,  225-236. 

Peer,  self,  and  teacher  ratings  of  elementary  school 
children's  personality  were  correlated  with  subsequent 
academic  performance.  For  the  peer  and  teacher  ratings, 
persistence,  independence,  interest,  and  nervousness 
correlated  significantly  with  academic  success. 


9.  Ballard,  M. , Reardin,  J.,  & Nelson,  L.  Student  and  peer 
ratings  of  faculty.  Teachinq  of  Psychology,  1976,  3, 
88-90. 

Student  and  peer  ratings  of  college  teachers' 
likeability  and  performance  correlated  well  with  each 
other. 


10.  Bartlett,  C.  J.  The  relationship  between  self-ratings  and 
peer  ratings  on  a leadership  behavior  scale.  Personnel 
Psychology,  1959,  12,  237-246 . 
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Self  and  peer  ratings  of  college  students'  leadership 
ability  were  collected  on  scales  featuring  both  a general 
dimension  of  leadership  potential  and  four  specific  leader- 
ship dimensions.  On  the  basis  of  factor  analysis  results, 
the  author  concluded  that  self  ratings  measure  only 
specific  leadership  dimensions  adequately,  while  peer 
ratings  are  applicable  for  assessing  both  overall  leader- 
ship ability  and  ability  in  specific  leadership  areas. 


11.  Bates,  A.  P.  Some  sociometric  aspects  of  social  ranking  in 
a small  face-to-face  group.  Soc iomet  ry , 1952,  JJ>,  330-3^1- 

Results  of  this  laboratory  experiment  indicated  that 
the  more  an  individual  behaves  according  to  group  norms, 
the  higher  he  will  be  rated  by  group  members. 

12.  Bayroff,  A.  G.  , Haggerty,  H.  R. , 6 Rundquist,  E.  A. 

Validity  of  ratings  as  related  to  rating  techniques  and 
conditions.  Personnel  Psychology,  1954,  ]_,  93 -11^. 

"Validity"  of  peer  rat inqs  made  by  Naval  officer 
students  was  evaluated  using  as  criteria  course  grades  and 
Officer  Classification  Test  scores.  Results  showed  that 
validity  was  enhanced  by  increasing  the  number  of  raters 
and  by  using  raters  with  higher  grades  and  test  scores. 
Neither  conditions  of  admi n i st rat  ion  (anonymity  vs.  identi- 
fication of  raters)  nor  the  type  of  rating  scale  (forced 
choice  vs.  graphic)  had  an  effect  on  validity. 

13.  Bell,  D.  B.,  6 Holz,  R.  F.  Summary  of  AR I research  on 
military  delinquency  ( AR I -RR- i 1 85) . Arlington,  Virginia: 
Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences,  1975- 

Peer  ratings  and  background  data  were  shown  to  predict 
military  delinquency. 

14.  Bentley,  R.  R. , S Rempel  , A.  M.  Peer  selection  vs.  expert 
judgment  as  a means  of  validating  a teacher  morale 
measuring  instrument.  Journal  of  Experimental  Education, 
1963,  31,  233-240. 
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Responses  to  teacher  morale  items  were  correlated 
separately  with  peer  ratings  of  morale  and  expert  judgments 
of  each'  teacher's  morale  for  item  analysis  purposes.  Expert 
judgments  produced  a questionnaire  with  the  greater  dis- 
crimination, and  responses  to  the  questionnaires  developed 
by  these  two  methods  correlated  moderately. 


15-  Berkshire,  J.  R. , 6 Nelson,  P.  0.  Leadership  peer  ratings 
related  to  subsequent  proficiency  in  training  and  in  the 
fleet  (58-20).  Naval  Aviation  Medical  Center,  Pensacola, 
Fior ida:  U.S.  Naval  School  of  Aviation  Medicine,  August, 

1958. 


Negative  peer  nominations  of  Naval  trainees  success- 
fully predicted  both  training  failure  and  fleet  assignment 
failure.  They  showed  more  validity  against  these  criteria 
than  did  the  positive  nominations. 


16.  Blackburn,  R.  T. , 6 Clark,  M.  J.  An  assessment  of  faculty 
performance:  Some  correlates  between  administrator, 

colleague,  student  and  self  ratings.  Sociology  of 
Educat  ion,  1975.  j«8,  2^2-258. 

Peer,  student,  and  supervisory  ratings  of  college 
teachers  correlated  highly  with  each  other,  but  showed  a 
near-zero  relationship  with  self  ratings. 


17.  Blaha,  W.  C.  A study  of  peer  and  reporting  senior  ratings 


■ — -i  - ^ :: — _ 2 

iii  a Marine  Corps  rifle  company.  Monterey, 
Nava  1 Postgraduate  School,  197**. 


Cal i forn i a : 


Supervisory  and  peer  ratings  of  Marine  Corps  enlisted 
performance  were  analyzed  in  a mul t i tra i t-mul t irater  matrix. 
There  was  no  evidence  of  convergent  or  discriminant 
validity  in  the  ratings. 


|8.  Blocher,  D.  H.  A multiple  regression  approach  to  predictin< 
success  in  a counselor  education  program.  Counse lor 
Education  and  Supervision,  198^,  j},  19"22. 
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A composite  consisting  of  peer  rankings  of  student 
counselor  effectiveness,  comprehensive  examination  scores, 
Kuder  Personal  Preference  Form  scores,  and  class  grades 
successfully  predicted  counselor  staff  ratings  of  effective- 
ness. 


19.  Booker,  G.  S.,  6 Miller,  R.  W.  A closer  look  at  peer 
ratings.  Personne  1 . 1 366  , k_3,  k2-k7. 

Peei  and  supervisory  ratings  of  the  promotab i 1 i ty  of 
military  cadets  correlated  highly,  and  the  two  sets  of 
ratings  taken  together  predicted  subsequent  military 
performance. 

20.  Borgatta,  E.  F.  The  stability  of  interpersonal  judgments 
in  independent  situations.  Journal  of  Abnormal  ind  Social 
Psychology , I960,  60,  1 88- 1 94 . 

Self  and  peer  ratings  of  assertiveness,  sociability, 
and  emotionality  were  consistent  across  different  situations 
and  group  sizes.  The  self  and  peer  ratings  correlated 
reasonably  highly,  with  the  magnitude  of  the  correlation 
varying  positively  as  a function  of  group  size. 

21.  Borgatta,  E.  F.  The  structure  of  personality  characteris- 
tics. Behavioral  Science,  196k,  9,  8-17. 

This  study  replicates  and  extends  the  findings  of 
Borgatta  (I960).  Two  additional  traits,  intelligence  and 
responsibility,  were  added  and  results  for  all  five  traits 
were  highly  similar  to  the  results  obtained  by  Borgatta 

(I960). 

22.  Borman,  W.  C.  The  rating  of  individuals  in  organizations: 

An  alternate  approach.  Organ i za t i one  1 Behavior  and  Human 
Per  formance , 197k,  1 05“  I 2k  . 

In  this  study  of  secretaries  and  their  supervisors,  the 
results  sugqest  that  peer  and  supervisory  ratings  are  based 
upon  different  perspectives  of  the  job  and  different  oppor- 
tunities to  view  job  behavior.  The  author  suggests  that  the 
convergent  validity  of  ratings  be  assessed  within,  not 
between,  organ i zat iona I levels. 
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23.  Brehm,  J.,  l Festinger,  L.  Pressures  toward  uniformity  of 
performance  in  groups.  Human  Relat ions,  1957,  10,  8 9 1 . 

This  paper  documents  the  existence  of  a rater  bias 
toward  uniformity  in  the  group  members'  ratings  of  each 
other  (central  tendency). 

2k.  Bryant,  G.,  6 Haack,  F.  Appraisal:  Peer-centered  and 
admin i st rator-centered . Educational  Leadership,  1977, 

3k,  608-612. 

This  paper  discusses  techniques  for  evaluating 
teacher  performance  including  peer  ratings. 

29.  Burke,  R.  J.  Some  preliminary  data  on  the  use  of  self- 
evaluations  and  peer  ratings  in  assigning  university 
course  grades.  Journal  of  Educational  Research,  1969,  62, 
kkk-kkB . ~~ 

Peer  ratings  of  students'  classroom  performance  showed 
high  interrater  reliability  and  high  correlations  with 
instructors'  ratings.  Self  ratings  were  more  favorable 
(lenient)  than  either  peer  or  instructor  ratings. 

26.  Buttery,  T.  J.,  I Williams,  W.  R.  CBTE:  Facilitating  the 
acquisition  of  • respons i b i I i t y . Teacher  Educator,  1977, 

J_2  . 23-26. 

This  paper  outlines  a system  (Competency  Based 
Teacher  Education)  for  teacher  education  that  incorporates 
peer  rat i ngs . 

27.  Carroll,  J.  B.  Ratings  on  traits  measured  by  a factored 
personality  inventory.  Journal  of  Abnormal  and  Social 
Psychology.  1952,  k_7.  626-632. 

Personality  self  ratings,  peer  ratings  and  test  scores 
of  Army  soldiers  were  i ntercorre I ated . Of  the  three  across- 
method  relationships,  sell  ratings  and  test  scores  correlated 
the  highest,  followed  by  self  ratings  and  peer  ratings,  and 
then  test  scores  and  peer  ratings.  Peer  rating  interrater 
reliabilities  were  satisfactory  and  were  found  to  be 
highest  for  the  more  observable  tiaits. 
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28. 


Centra,  J.  A.  Colleagues  as  raters  of  classroom 
Journal  of  Higher  Education,  1975,  ^6,  327*337* 


instruct  ion. 


Peer  ratings  of  college  teachers'  performance  showed 
good  test-retest  reliability  but  poor  interrater  reliability. 
Also,  the  peer  ratings  contained  leniency  error,  and 
correlations  between  peer  and  student  ratings  were  low. 


29*  Centra,  J.  A.  The  how  and  why  of  evaluating  teaching. 

New  Directions  for  Higher  Education,  1977,  1 7 , 93“ 106. 

This  paper  discusses  teacher  evaluation  technigues. 

The  author  concludes  that  peer  ratings  of  teachers  may 
suffer  from  low  reliability  and  are  therefore  best  used 
for  diagnostic  purposes  rather  than  for  personnel  decisions. 


30.  Cox,  J.  A.,  6 Krumboltz,  J.  D.  Racial  bias  in  peer  ratings 
of  basic  airmen.  Soc iometry , 1958,  2J_,  292*299* 

Some  evidence  of  racial  bias  was  found  in  leadership 
peer  ratings  of  Air  Force  basic  trainees.  Individuals 
tended  to  rate  members  of  their  own  race  higher  than 
members  of  the  other  race,  but  there  was  agreement  between 
races  regarding  the  rank-order  of  leadership  ability  within 
each  of  the  racial  groups. 

31*  Criswell,  J.  H.  Sociometric  concepts  in  personnel 
admi n is t ra t ion.  Soc iometry , 19^9,  1 2 , 287*300. 

This  paper  describes  how  sociometric  techniques, 
including  peer  judgments,  might  be  applied  in  military  and 
industrial  settings.  Basic  concepts  and  statistical  prob- 
lems related  to  sociometrics  are  discussed. 


32.  D'Augelli,  A.  R.  Group  composition  using  interpersonal 
skills:  An  analogue  study  on  the  effects  of  members' 
interpersonal  skills  on  peer  ratings  and  group  cohesiveness. 
Journal  of  Counseling  Psychology,  1973,  20^,  531*53^- 
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In  an  experimental  setting,  college  student  group 
members  with  highly  rated  interpersonal  skills  received 
higher  peer  ratings  on  dimensions  of  understanding,  honesty, 
openness,  and  acceptance  than  members  with  lower  rated 
levels  of  interpersonal  skills.  Also,  the  cohesiveness  of 
groups  with  members  having  high  interpersonal  skills  was 
judged  greater  than  the  cohesiveness  of  groups  with  members 
lower  in  interpersonal  skills. 


33-  DeJung,  J.  E.  Effects  of  rater  frames  of  reference  on  peer 
ratings.  Journal  of  Experimental  Education,  1964,  33. 
121-131. 

Children  given  an  explicit  frame  of  reference  on  which 
to  base  their  ratings  (i.e.,  "compared  to  all  persons 
known"),  made  less  positive  ratings  than  did  children  given 
no  frame  of  reference.  However,  frame  of  reference  did  not 
affect  the  rank  order  of  their  peer  ratings. 


34.  DeJung,  J.  E.,  6 Kaplan,  H.  Some  differential  effects  of 
race  of  rater  and  ratee  on  early  peer  ratings  of  combat 
aptitude.  Journal  of  Applied  Psychology,  1962,  46^,  370-374. 

Peer  ratings  of  the  combat  potential  of  Army  recruits 
were  examined  for  racial  bias.  A hypothesis  that  raters 
give  higher  ratings  to  members  of  their  own  race  than  to 
members  of  a different  race  was  supported  only  in  the  case 
of  the  black  recruits. 


35.  DeLeon,  P.  H.,  DeLeon,  J.  L.,  £ Swihart,  P.  J.  Relation  of 
accuracy  of  self-perception  and  peer  ratings.  Perceptual 
and  Motor  Skills,  1969,  2J3,  966. 

Males  providing  accurate  self  ratings  (evaluated  by 
extent  of  peer-self  rating  agreement)  were  rated  more 
positively  by  their  peers  on  sociability  than  those  with 
innacurate  self  perceptions.  This  effect  was  not  observed 
for  females. 


! : 


36.  Dilley,  J.  S.  Supervisory  ratings  of  counselor  trainees 
in  a simulated  work  setting  as  compared  with  peer  and 
instructor  ratings  on  the  same  trainees  in  an  academic 
setting.  Counselor  Education  and  Supervision,  I96*t,  3, 
70-73- 

Ratings  of  student  counselor  effectiveness  by  peers, 
instructors,  and  field  supervisors  showed  high  agreement. 

'37.  Doll,  R.  E.  Officer  peer  ratings  as  a predictor  of  failure 
to  complete  flight  training.  Aerospace  Medicine,  1963,  3^, 

130-131. 

Naval  cadets  and  officer  students  rated  their  peers' 
performance  in  training.  The  ratings  predicted  subsequent 
school  dropout  to  a moderate  degree.  Ratinqs  were  also 
combined  with  objective  aptitude  test  scores,  with  peer 
ratings  adding  to  the  validity  of  the  predictors  only  for 
the  cadets.  The  author  explains  this  latter  finding  by 
noting  that  cadets  are  in  position  to  observe  a wider  range 
of  their  peers'  behaviors  than  are  officer  students. 

38.  Doll,  R.  E.,  f.  Longo,  A.  E.  Improving  the  predictive 

effectiveness  of  peer  ratings.  Personnel  Psychology,  1962. 

J_5,  215-220. 

Regarding  peer  evaluations  of  Naval  Training  Cadets, 
the  validity  of  low  nominations  as  predictors  of  training 
failure  was  enhanced  by  using  nominations  only  on  the 
per formance-or i ented' d imens ions , removing  from  consideration 
ratings  on  those  dimensions  related  to  "antisocial 
behav ior . " 


39.  Dooley,  D.  Assessing  nonprofessional  mental  health  workers 
with  the  GAIT.  An  evaluation  of  peer  ratings.  Amo i i c a n 
Journal  of  Community  Psychology,  1975.  3_,  99"  1 10- 

Applicants  for  a mental  health  counselor  job  were 
rated  by  their  peers  on  interpersonal  skills  displayed  in 
dyadic  situations  (the  technique  was  called  Group  Assess- 
ment of  Interpersonal  Trai ts--"GAIT") . Interrater 
reliability  of  the  ratings  was  poor  and  the  scales  correlated 
highly. 
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*40.  Downey,  R.  G.  Note  on  the  Kaufman  and  Johnson  studies  of 
the  differential  validities  of  peer  nomination  techniques. 
Journal  of  Applied  Psychology,  1975,  60,  2*45-2*46. 

This  note  presents  a methodological  critique  of  the 
Kaufnan  and  Johnson  (I97*»)  study.  It  is  concluded  that 
comparative  validity  of  different  scaling  methods  for  peer 
nominations  is  an  issue  not  yet  fully  clarified. 

*4 1 . Downey,  R.  G.,  Medland,  F.  F.,  £ Yates,  L.  G.  Evaluation 
of  a peer  rating  system  for  predicting  subsequent  promotion 
of  senior  military  officers.  Journal  of  Applied  Psychology, 
1976,  6J_,  206-209. 

Peer  ratings  of  the  promotabi 1 i ty  of  Army  Colonels 
showed  moderate  validity  in  predicting  subsequent  promotion 
to  General,  with  these  results  stable  across  career  groups 
and  levels  of  education.  The  authors  noted  some  resistance 
to  the  peer  rating  procedure  on  the  part  of  the  raters. 

42.  Dugan,  R.  D.  Comparison  of  evaluation  of  B-29  crews  in 
training  and  in  combat.  American  Psychologist,  1953,  8, 
343-344. 

Peer  ratings  of  the  combat  proficiency  of  airplane 
crews  were  used  as  criteria  for  assessing  the  predictive 
validity  of  proficiency  tests.  A significant  correlation 
was  obtained  between  test  scores  and  criteria.  In  addi- 
tion, peer  ratings  obtained  in  training  corresponded  well 
with  peer  ratings  obtained  later  in  a combat  situation. 

43.  Eckard,  P.  J.,  5 McElhinney,  J.  H.  Teacher  evaluation  and 
educational  accountability.  Educational  Leadership,  1977, 
34.  613-618. 

This  paper  proposes  a teacher  evaluation  system  that 
incorporates  peer  ratings. 
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*4*4.  Edwards,  R.  C.  Personal  traits  and  "success"  in  schooling 
and  work.  Educational  and  Psychological  Measurement,  1977, 
.37,  125-138. 

Personality  peer  ratings  of  students  and  working 
personnel  were  shown  to  possess  high  interrater  reliability 
and  successfully  predicted,  respectively,  school  and  work 
performance.  Ratings  were  factor  analyzed,  with  different 
factors  showing  different  degrees  of  predictive  validity. 

*45.  Elliott,  L.  L.  Factorial  structure  of  airman  self-ratings 
and  their  relationship  to  peer  nominations  (WADD-TN-60- 1 I , 
AD-2A2  388).  LackT  and  Air  Force  Base,  Texas:  Personnel 
Laboratory,  Wright  Air  Development  Division,  July  I960. 

Factor  analyses  of  self  ratings  and  peer  nominations 
showed  some  similarity  in  structure,  but  self  ratings 
appeared  to  have  a larger  number  of  underlying  dimensions. 
Self  ratings,  years  of  education,  and  Armed  Forces  Quali- 
fying Test  scores  each  correlated  positively  with  peer 
nominat ions  of  619  basic  airmen.  The  author  concludes  that 
peer  nominations  show  more  promise  than  self  ratings  for 
purposes  of  assessment. 

*46.  Engle,  K.  B.,  & Betz,  R.  L.  Peer  ratings  revisited. 

Counselor  Education  and  Supervision,  1971,  J_0,  165-170. 

Peer  and  supervisory  rankings  of  student  counselor 
effectiveness  correlated  highly,  and  this  relationship 
was  particularly  strong  for  NDEA  Institute  groups,  pre- 
sumably due  to  a relatively  high  degree  of  exposure  to 
other  counselors  in  the  Institute  program. 


*47.  Ewart,  E.  S.  Factorial  structure  of  airman  peer  nomina- 
tions^ (WADD-TN-60- I *40,  AD-2I4)  1425).  Lackland  Air  Force 
Base,  Texas:  Personnel  Laboratory,  Wright  Air  Development 
Division,  June  I960. 

Peer  ratings  of  Air  Force  recruits  were  factor 
analyzed,  yielding  four  interpretable  factors:  a general 
factor  of  acceptance  or  rejection  by  peers,  good 
naturedness,  sociability,  and  motivation  for  military  life. 
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Farley,  J.  A.,  & Mayfield,  E.  C.  Peer  nominations  without 
peers?  Journal  of  Applied  Psychology,  1976,  6J_,  109-111. 

"Self-peer"  ratings  (Holmes  & Tyler,  1968)  showed  no 
validity  in  predicting  sales  performance  of  life  insurance 
agents . 


Fiske,  D.  W.  Consistency  of  the  factorial  structures  of 
personality  ratings  from  different  sources.  Journal  of 
Abnormal  and  Social  Psychology,  19^9,  Vj_,  329*3^ . 

Self  ratings,  peer  ratings,  and  assessor  ratings  of 
college  applicants  showed  a similar  factor  structure  of 
four  factors:  social  adaptability,  emotional  control, 
conformity,  and  inquiring  intellect.  An  additional  factor, 
named  confident  self  expression,  was  discovered  in  the  self 
and  peer  ratings  only. 


Fiske,  D.  W.  Variability  among  peer  ratings  in  different 
situations.  Educational  and  Psychological  Measurement, 
I960,  20,  283-292. 

Peer  and  observer  ratings  of  individuals'  behavior 
across  different  problem-solving  groups  were  evaluated  to 
assess  consistency  of  ratings  across  groups.  Results 
showed  that  subjects  received  inconsistent  ratings  across 
sessions,  and  this  inconsistency  was  attributed  to 
interrater  unreliability  rather  than  inconsistency  in 
behavior . 


Fiske,  D.  W. , & Bourne,  E.  J.  Thresholds  for  attributing 
can  affect  factorial  structure.  Educat iona 1 and 
Psychological  Measurement,  1977,  37,  71 3" 7 2 3 


Subjects  provided  personality  peer  ratings  under  one  of 
two  instructional  sets:  (a)  to  select  traits  that 
"definitely  and  without  question  apply"  to  the  ratee,  or  (b) 
to  select  traits  that  "might  reasonably  be  applied"  to  the 
ratee.  Ratings  administered  under  condition  (a)  yielded  a 
factor  structure  that  more  closely  [than  (b)]  resembled  the 
structure  found  by  Tupes  and  Christal  (1958)  among  others. 


52.  Fiske,  0.  W. , 6 Cox,  J.  A.  The  consistency  of  rating  by 
peers.  Journal  of  Applied  Psychology,  I960,  11-17- 

This  study  investigated  in  an  experimental  setting  the 
effects  of  several  factors  on  peer  ratings:  (a)  the  kind 
of  task  assigned  to  the  group;  (b)  the  group  to  which  a 
person  is  assigned;  (c)  the  frequency  of  contact  between 
group  members;  (d)  the  nature  of  the  trait  being  rated;  and 
(e)  the  role  of  the  rater  (peer  or  observer).  The  group  to 
which  a ratee  was  assigned  affected  peer  ratings.  Also, 
peer  ratings  tended  to  be  more  positive  than  observer 
ratings,  and  results  suggest  that  the  level  of  interrater 
agreement  depends  upon  the  trait  being  rated. 


53-  Fiske,  D.  W. , Cox,  J.  A.,  t van  der  Veen,  F.  Cons i stency 
and  variability  in  peer  ratings  [WADC-TR-59-37 ( 1 , 1 1 ) , 

AD-215  **33;  AD-215  **8*0  . Lackland  Air  Force  Base,  Texas: 
Personnel  Laboratory,  Wright  Air  Development  Center,  May 
1959- 

In  an  experimental  setting,  the  effects  of  the  following 
variables  on  observer  and  peer  ratings  were  investigated: 
nature  of  the  group's  task  (verbal  or  physical);  the  group 
to  which  subjects  were  assigned;  duration  of  contact; 
nature  of  the  trait  rated  (evaluative  vs.  descriptive);  and 
the  rater's  role  (peer  or  observer).  Only  the  group 
assigned  affected  peer  (and  observer)  ratings,  with  between- 
group  rating  reliability  lower  than  within-group 
reliabilities. 


5A.  Flyer,  E.  S.,  Barron,  E. , C Bigbee,  L.  Pi screpanc ies 


t 

Base,  Texas:  Human  Resources  Research  Center,  September  1953- 

This  study  showed  that  for  a group  of  Air  Force 
personnel  self-perception  (how  one  rates  self),  social 
perception  (how  one  believes  peers  rate  him),  and  group 
perception  (how  peers  actually  rate  him)  judgments  did  not 
agree  closely.  Group  perceptions  agreed  more  closely  with 
social  perceptions  than  sel f-percept ions , suggesting  that 
peer  ratings  may  correspond  more  to  how  persons  believe 
others  see  them  than  to  how  they  see  themselves. 


between  self-descriptions  and  group  ratings  as  measures  of 
ins ight~  Research  Bulletin  ( 53- 33 ) • Lackland  Ai r Force 
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55. 


Flyer,  E.  S. , S Bigbee,  L.  R.  Primary  flying  grades, 
pilot  stanine,  and  preflight  peer  nominations  as  predictors 
of  basic  pilot  training  criteria  (PRL-TM-55- 1 7) • Lackland 
Air  Force  Base,  Texas:  Air  Force  Personnel  and  Training 
Research  Center,  June  1 955 • 

Peer  nominations  of  Air  Force  student  pilots  success- 
fully predicted  training  success/failure.  However,  the  peer 
nominations  did  not  predict  success  in  training  as  well  as 
did  pilot  aptitude  test  scores  and  instructors'  ratings 
taken  together. 

56.  Folkins,  C.,  & Spensely,  J.  Peer  rating  by  a community 
mental  health  team:  A positive  approach  to  accountability. 
American  Journal  of  Ortho-psychiatry,  1977,  *»7 . 331~335. 

Community  mental  health  team  members  rated  their  peers 
on  interpersonal  skills  and  job  competence.  The  ratings 
possessed  high  interrater  reliability  and  successfully 
predicted  supervisory  ratings.  Also,  team  member  partici- 
pants responded  favorably  to  the  peer  rating  procedures. 

57.  Freeberg,  N.  E.  Relevance  of  rater-ratee  acquaintance  in 
the  validity  and  reliability  of  ratings.  Journal  of 
Applied  Psychology,  1 969 , 5j[,  513-52*4. 

In  an  experimental  setting,  two  variables  of  rater- 
ratee  relationships  were  manipulated:  (a)  relevance  vs. 
non- re  I evance  of  the  behaviors  observed;  and  (b)  visual 
only  vs.  more  extended  contact  with  other  group  members. 
Results  suggest  that  relevance  of  the  behaviors  observed 
is  important  for  validity  of  peer  ratings,  but  that 
impressive  reliability  (interrater)  may  be  obtained  with 
peer  ratings  based  on  visual  information  only.  Results 
also  suggest  that  validity  is  enhanced  by  more  extended 
contact  with  other  group  members. 

58.  Golding,  S.  L.,  6 Knudson,  R.  M.  Mul t i var iable-mul t imethod 
convergence  in  the  domain  of  interpersonal  behavior. 
Multivariate  Behavioral  Research,  1975.  10, 
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A factor  analysis  of  personality  self  report  inven- 
tories, self  ratings,  and  peer  ratings  revealed  three  second 
order  factors:  aggressive  dominance,  affiliation- 
sociability,  and  autonomy.  Convergent  validity  for  these 
dimensions  was  relatively  high  across  all  three  measurement 
methods. 


59.  Gordon,  L.  V.  Estimating  the  reliability  of  peer  ratings. 

Educational  and  Psychological  Measurement  , 1969,  29, 
305-313-  ~ 

This  paper  discusses  the  use  of  coefficient  alpha  for 
assessing  the  reliability  of  peer  ratings.  Suggestions  are 
made  about  appropriate  applications  of  various  reliability 
formulae  to  different  types  of  rating  data. 

60.  Gordon,  L.  V.,  & Medland,  F.  F.  The  cross  group  stability 
of  peer  ratings  of  leadership  potential.  Personnel 
Psychology,  1904,  _1J3,  1 73-178. 

This  study  found  that  peer  ratings  of  Army  enlisted 
personnel  tended  to  be  stable  across  different  groups  of 
peers . 

61.  Gordon,  M.  E.  The  effect  of  the  correctness  of  the 
behavior  observed  on  the  accuracy  of  ratings.  Organ  i za- 
tional  Behavior  and  Human  Performance,  1970,  5^,  3^6-377 • 

Life  insurance  managers  rated  the  filmed  performance 
of  an  agent  completing  a telephone  transaction.  Judgments 
of  behavior  against  a correct  standard  indicated  a 
"Differential  Accuracy  Phenomenon"  (DAP) -- i ncorrect 
behaviors  were  identified  with  less  accuracy  than  were 
correct  behaviors.  Further  experience  with  the  rating  task 
did  increase  overall  accuracy,  but  the  DAP  remained. 

62.  Gormly,  J.,  & Edelberg,  W.  Validity  in  personality  trait 
attribution.  American  Psychologist,  1974,  29,  189-193- 

College  students'  peer  ratings  of  aggressiveness 
correlated  highly  with  ratings  of  aggressiveness  by  judges 
who  observed  them  in  an  interpersonal  situation.  Self- 
ratings  showed  little  agreement  with  peer  ratings. 
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63.  Grande,  P.  P.  The  use  of  self  and  peer  ratings  in  a Peace 
Corps  traininq  program.  Vocational  Guidance  Quarterly, 
1966,  J_4,  244-246.' 

Peer  effectiveness  ratings  of  Peace  Corps  trainees 
successfully  predicted  subsequent  ratings  of  on-site 
performance  made  by  a Peace  Corps  Project  Director. 

64.  Guilford,  J.  P. , Christensen,  P.  R. , Toffe,  G.,  S 
Wilson,  R.  C.  Ratings  should  be  scrutinized.  Educat  iona 1 
and  Psychological  Measurement,  1962,  2_2,  439-447. 

This  paper  discusses  difficulties  arising  when  ratings 
are  used  as  criteria  for  validating  tests. 

65.  Gunderson,  E.  K.  E.,  £ Nelson,  P.  D.  Criterion  measures 
for  extremely  isolated  groups.  Personnel  Psychology, 

1966,  29.  67-60. 

Moderate  to  high  correlations  were  obtained  between 
peer  and  supervisory  ratings  of  personnel  at  an  Antarctic 
sc i ent i f i c s t at  ion . 


66.  Guthrie,  E.  R.  The  evaluation  of  teaching.  Educational 
Record , 1949.  30,  109- I 15. 

This  paper  concludes  that  student  and  peer  ratings  are 
the  best  performance  appraisal  methods  for  evaluating 
teachers . 


67.  Hake),  M.  D.  Normative  personality  factors  recovered  f rom 
ratings  of  personality  descriptors:  The  beholder's  eye. 
Personnel  Psychology.  1974,  2_7,  409-421. 

College  students  performed  two  rating  tasks:  (a) 
making  personality  ratings  of  a person  they  knew  but  who  was 
not  physically  present,  and  (b)  rating  the  similarity  in 
meaning  of  personality  descriptors.  Factor  analyses  in 
both  cases  revealed  the  recurring  five  factor  structure 
identified  by  Tupes  and  Christal  (1968),  thus  presenting 
strong  evidence  that  peer  ratings  reflect  raters' 

"implicit  personality  theories." 
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68.  Heifer,  R.  E.  Peer  evaluation:  Its  potential  usefulness  in 
medical  education.  British  Journal  of  Medical  Education, 

1972,  6,  224-231. 

This  paper  reviews  peer  rating  literature  and 
discusses  possible  applications  of  peer  ratings  in  a 
medical  context. 


69.  Hoffman,  E.  L. , 6 Rohrer,  J.  H.  An  objective  peer  evalua- 
tion scale:  Construction  and  validity.  Educational  and 
Psychological  Measurement,  1954,  1 4 , 332-341  . 

This  paper  discusses  the  development  and  application 
of  peer  rating  scales  for  evaluating  Marine  Corps  officer 
candidates.  Peer  ratings  successfully  predicted  subseguent 
promotions  within  platoon. 


70.  Hollander,  E.  P.  Authoritarianism  and  leadership  choice 
in  a military  setting.  Journal  of  Abnormal  and  Social 
Psychology,  1954,  49,  365-370. 

Leaders  selected  by  peer  nomination  in  a military 
setting  tended  to  be  non-authoritarian  in  leadership  style. 
Also,  intelligence  correlated  positively  with  the  peer 
nominations.  Leadership  styles  of  the  raters  did  not 
appear  to  influence  their  ratings. 


71.  Hollander,  E.  P.  Buddy  ratings:  Military  research  and 
industrial  applications.  Personnel  Psychology,  1954,  7, 
385-393. 

This  paper  summarizes  early  research  on  the  quality, 
techniques,  and  applications  of  peer  ratings.  It  focuses 
on  military  research  and  possible  applications  of  peer 
ratings  for  predicting  performance  in  private  industry. 

72.  Hollander,  E.  P.  Peer  nominations  on  leadership  as  a 
predictor  of  the  pass- fail  criterion  in  naval  air  training. 
Journal  of  Applied  Psychology,  1954,  38,  150-153- 


-- 


Peer  nominations  of  the  leadership  potential  of  Naval 
Aviation  Cadets  successfully  predicted  performance  criteria 
not  directly  related  to  leadership. 


73-  Hollander,  E.  P.  The  friendship  factor  in  peer  nominations. 
Personnel  Psychology,  1956,  9,  ^4  3 5 ~ 7 . 

Peer  ratings  of  candidates  for  Naval  Officer  Candidate 
School  were  analyzed  for  the  possible  existence  of  a 
"friendship  factor."  Results  suggested  a positive  relation- 
ship between  popularity  and  peer  nomination  scores,  but  it 
was  concluded  that  this  relationship  does  not  significantly 
alter  the  predictive  validity  of  peer  ratings. 

1h.  Hollander,  E.  P.  Interpersonal  exposure  time  as  a determi- 
nant of  the  predictive  utility  of  peer  ratings. 

Psychological  Reports,  1956,  2^,  445-^8. 

This  study  found  that  peer  ratings  of  candidates  for 
Naval  OCS,  gathered  after  a 3~week  acquaintance,  were  highly 
reliable  and  successfully  predicted  training  performance. 

The  ratings  after  3 weeks  acquaintance  did  not  differ 
significantly  from  those  gathered  after  the  groups  had  been 
together  for  6 weeks. 


75.  Hollander,  E.  P.  The  reliability  of  peer  nominations  under 
various  conditions  of  administration.  Journa 1 of  Appl i ed 
Psychology,  1957,  A 1 , 85”90. 

This  study  investigates  the  effects  of  three  different 
administration  conditions  on  the  reliability  of  peer  ratings 
made  by  candidates  for  Naval  OCS:  the  period  of  time  the 
group  has  spent  together,  the  nature  of  the  set  imposed  on 
the  raters  (administrative  vs.  for  research  only),  and  the 
qual i ty/character i st ic  to  be  rated.  Reliable  peer  ratings 
were  obtained  with  groups  together  for  even  relatively  short 
time  intervals  (I  week).  Rating  set  had  no  effect.  Some 
qualities  rated  (e.g.,  leadership  quality)  showed  greater 
stability  over  time  than  did  others  (e.g.,  success  in  OCS). 
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76. 


Hollander,  E.  P.  Va I idity  of  peer 
a distant  performance  criterion 
Psychology,  [965,  A9,  ^ 3 - A 3 8 . 


nominations  in  predicting 
Journal  of  Applied 


Measures  of  Naval  officer  performance  were  correlated 

years  ear!  ie^,n9Th9athered  °CS  traini"9  several 

years  earlier.  The  peer  ratings  successfully  predicted 
posttraining  performance. 


77.  Hollander.  E.  P.  , £ Sausser,  E.  R.  A further  cons  idem  i inn 
pfogran • "Tr'n 

Predict  ion  ol  completion  or  fa  i I u7T> 1 

Pensaco  I a . 

1953  001  °f  Aviation  M^icine,  October 


Both  peer  nominations 
preflight  school  grades  of 
ful ly  predicted  f a i lure  in 


of  leadership  potential  and 
Naval  Aviation  Cadets  success- 
Nava I air  training. 


78.  Hollander  E.  P. , £ Webb,  W.  B.  Leadership,  followership 

of  f 'Pj  PAn.ana,ysis  of  peer  nominations.  Journal 

of  Abnormal  and  Social  Psychology.  1955,  50  ( 163-167-; 

fn  .U*!ng  pee:  nominations  of  Naval  Air  Cadets,  it  was 
ound  that  ratings  of  leadership  and  "followership" 

corrPl*?  H }tV  effeCtivt'  follower)  were  highly 

“er'"':„d  Z°?n‘r’r>:  ln  3dd  1 1 ion.  rating,  of  friendship 

were  shown  to  be  significantly  more  highly  related  to 
followership1  than  to  leadership. 


79. 


Holmes,  0.  S.  Conscious  self-appraisal 
motivation:  The  self-peer  rank  method 
of  Consulting  and  Clinical  Psychology. 


of  achievement 
revisited.  Journal 
197),  36.  23^ 


’ L ^ ranKing  method  (Holmes  f.  Tyler,  1963)  J. 
described,  and  rat  mgs  of  achievement  motivation  using  t h i • 
method  successfully  predicted  academic  performance. 


80.  Holmes,  0.  S.,  £ Tyler.  J.  D.  Direct 
measurement  of  achievement  motivation 

Psychology . I 968” 


Consulting  and  Clinical 


versus  projective 
Journal  of 
32,' 712-7  I 7. 
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Need  for  achievement  was  assessed  by  the  Thematic 
Apperception  Test,  self-reports,  and  "self-peer"  ratings. 
The  self-peer  rating  procedure  required  subjects  to  list 
friends  and  then  to  rate  themselves  relative  to  each 
friend.  Only  the  "self-peer"  ratings  correlated 
significantly  with  classroom  grades. 


81 . Izard,  C.  E.  Personality  correlates  of  sociometric  status. 
Journal  of  Applied  Psychology,  1959,  ^_3_,  89-93- 

Peer  ratings  of  military  personnel  on  a global 
dimension  of  leadership  correlated  significantly  with 
personality  tests,  physical  aptitude  tests,  and  supervisory 
ratings.  There  was  a negative  relationship  observed  between 
ratings  received  from  peers  and  tendencies  toward  psychoso- 
matic ailments,  with  a positive  relationship  observed 
between  peer  ratings  and  participation  in  group  activities. 

82.  Izard,  C.  E.,  & Rosenberg,  N.  Prediction  of  peer  leadership 
ratings  by  a forced-choice  test  under  various  conditions. 
American  Psychologist,  195^,  9^,  397. 

A test  of  leadership  potential  for  military  officers 
showed  a relatively  low,  but  positive,  correlation  with  peer 
ratings  of  leadership  potential. 

83.  Jansen,  D.  G.,  Robb,  G.  P. , & Bonk,  E.  C.  Peer  ratings  of 
practicum  counselors.  Journal  of  Counseling  Psychology , 

1972,  Jj_9,  333-339- 

Peer  ratings  of  student  counselors  on  three  dimensions 
(competence,  knowledge,  and  likeability)  were  highly 
correlated.  Counselors  who  were  rated  higher  tended  to 
score  higher  on  aptitude  tests,  score  more  favorably  on 
personality  inventories,  and  make  better  grades  than  did 
counselors  who  were  rated  lower. 


8^.  Jansen,  D.  G. , Robb,  G.  P.,  t Bonk,  E.  C.  Peer  ratings  and 
self  ratings  on  twelve  bipolar  items  of  practicum  counselors 
ranked  high  and  low  in  competence  by  their  peers.  Journa 1 
of  Counseling  Psychology,  1973,  2_0,  1419-^2*4. 
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Peer  rankings  were  used  to  form  high  and  low  competence 
groups  of  student  counselors.  The  high  competence  group 
members  were  rated  more  favorably  by  their  peers  on  several 
personality  dimensions,  while  no  such  differences  were 
found  for  the  self  ratings. 


85.  Kane,  J.  S.,  t Lawler,  E.  E.  Methods  of  peer  assessment. 
Psychological  Bulletin,  1978,  8_5,  555-586. 

This  review  of  research  on  peer  nominations,  peer 
ratings,  and  peer  ranking  concludes  that  peer  nominations 
show  the  highest  reliability  and  validity,  with  peer 
rat ings  having  merit  for  feedback  purposes.  It  is 
recommended  that  peer  assessments  be  combined  with  other 
performance  measures  to  yield  the  highest  quality  per- 
formance evaluations. 


86.  Kaufman,  G.  G.  Comments  on  Downey's  note:  Discussion  and 
further  analysis  of  the  differential  validities  of  peer 
nomination  scales.  Journal  of  Applied  Psychology,  1975. 
60,  247-248 . 

This  paper  defends  the  Kaufman  and  Johnson  (1974) 
study,  in  light  of  criticisms  by  Downey  (1975). 
Methodological  issues  raised  by  Downey  are  discussed,  and 
it  is  concluded  that  these  issues  are  irrelevant  to  the 
interpretation  of  the  Kaufman  and  Johnson  study. 


87.  Kaufman,  G.  G. , & Johnson,  J.  C..  Scaling  peer  ratings:  An 
examination  of  the  differential  validities  of  positive  and 
negative  nominations.  Journal  of  Applied  Psychology,  1974, 
59.  302-306. 

This  study  examined  correlations  between  peer  evalua- 
tions and  performance  of  Reserve  Officers'  Training  Corps 
cadets.  Results  showed  that  positive  nominations  correlated 
more  highly  with  performance  than  did  negative  nominations. 
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88.  Keisler,  E.  R.  Peer  group  ratings  of  high  school  pupils 
with  high  and  low  school  marks.  Journal  of  Experimental 
Education,  1955,  23.,  375-378. 

High  school  students  with  high  academic  grades  were 
rated  more  positively  by  their  peers  on  behavior  oriented 
scales  related  to  academic  success  and  on  personality 
related  scales  than  were  students  with  low  academic 
achievement. 


89.  Klimoski,  R.  J.,  £ London,  M.  Role  of  the  rater  in 

performance  appraisal.  Journal  of  Applied  Psychology, 

1 974.  59,  445-451.  

Th i s study  investigated  supervisory,  peer,  and  self 
ratings  of  registered  nurses.  Factor  analysis  of  the 
ratings  from  the  three  sources  suggested  that  raters  in 
different  positions  used  different  criteria  on  which  to 
base  their  ratings,  and  that  an  important  source  of  bias 
in  ratings  stems  from  the  position  of  the  rater  relative 
to  the  person  being  rated. 


90.  Klockar;,  A.  J.  Personality  variables  related  to  peer 
selection.  Educational  and  Psychological  Measurement, 

1978,  38,  513-517. 

Peer  ratings  of  potential  as  a resident  assistant  in 
a college  dormitory  correlated  significantly  with  scores  on 
several  self-report  personality  test  scales,  always  in  the 
socially  desirable  direction. 


91.  Korman,  A.  K.  The  prediction  of  managerial  performance:  A 
review.  Personnel  Psychology,  1968,  2J_,  295- 322 . 

In  this  literature  review,  the  author  concludes  that 
peer  ratings  show  good  promise  for  predicting  the  perfor- 
mance of  managers. 

92.  Kraut,  A.  I.  Prediction  of  managerial  success  by  peer  and 
training  staff  ratings.  Journal  of  Applied  Psychology, 

1975,  60,  14-19- 
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This  study  examined  the  predictive  validity  of  peei  and 
training  staff  ratings  of  industrial  managers  and  executives. 
It  was  concluded  that  peer  ratings  obtained  in  a training 
course  are  successful  predictors  of  performance  for  high 
level  business  managers,  more  successful  than  training 
staff  ratings. 


93.  Kubany,  A.  J.  Use  of  sociometric  peer  nominations  in 

medical  education  research.  Journal  of  Applied  Psychology, 
1957.  389-  39't. 

This  paper  discusses  problems  related  to:  (a)  factor 
analysis  of  peer  nomination  variables;  (b)  assessing  the 
reliability  of  peer  ratings;  and  (c)  the  effects  of  infre- 
quent rater-ratee  contact  on  peer  ratings.  Also,  results 
of  a study  reported  in  the  paper  showed  that  medical  student 
peer  ratings  successfully  predicted  course  grades. 

9((.  Landy , F.  J.,  & C,uion,  R.  M.  Development  of  scales  for  the 
measurement  of  work  motivation.  Organ i zat iona I Behavior 
and  Human  Performance,  ld/o,  5,  93-103. 

This  paper  discusses  the  development  of  behavioral ly 
anchored  rating  scales  foi  use  by  engineers  in  peer  assess- 
ment of  work  motivation.  The  scales  showed  good  interrater 
reliability,  and  the  authors  concluded  that  such  motivation 
scales  may  he  appropriate  for  a wide  range  of  occupations. 

95.  Lawler,  E.  E.  The  mul t i t ra i t-mu I t i ra t er  approach  to 
measuring  managerial  job  performance.  Journa I of  App I i ed 
Psychology , 1967,  5^,  369- 38 1 . 

This  paper  presents  a technique  for  analyzing  super- 
visory, peer,  and  self  ratings  in  a nui  1 1 i tra i t -mul t iratei 
matrix.  The  author  applied  the  technique  to  ratings  of 
managers,  ami  it  was  concluded  that  the  matrix  concept 
offers  promise  for  assessing  the  convergent  and  discriminant 
validities  of  ratings. 

96.  Lewin,  A.,  Dubno,  °.  , f.  Akulo,  W.  Face-to-face  interaction 
in  the  peer  nomination  process.  Journal  of  Applied 
Psychology , 1971,  56,  1(95-1(97. 
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Peer  ratings  of  individual  contributions  to  a 
laboratory  group's  task  performance  were  highly  correlated 
across  two  conditions:  (a)  a condition  in  which  peers 
interacted  with  other  group  members  and  (b)  a condition  in 
which  raters  viewed  a videotape  of  the  group  interacting. 


97.  Lewin,  A.  Y.  , & Layman,  S.  S.  Information  processing  models 
of  peer  nominations  (TR-4,  1978,  TR-3,  1977,  TR-2,  1977) . 
Durham,  North  Carolina:  Graduate  School  of  Business 
Administration,  Duke  University. 

This  study  investigated  the  decision  processes  under- 
lying peer  ratings.  Videotapes  of  group  sessions  were  used 
to  develop  scoring  procedures  based  on  verbal  and  non- 
verbal behavior  in  five  categories:  mutual  influencing, 
categor iz i ng/summar i z i ng,  soc ia 1 -d i rect ive , quantity  of 
verbal  communication,  and  listening.  Rankings  of  group 
members'  performance  based  on  these  scoring  procedures 
correlated  highly  with  actual  peer  ratings,  indicating  some 
understanding  of  the  cues  peers  use  in  making  evaluations. 

98.  Lewin,  A.  Y. , & Zwany,  A.  Peer  nominations:  A model, 
literature  critique,  and  a paradigm  for  future  research. 
Personnel  Psychology,  1976,  2^,  4 2 3 _ 7 . 

The  authors  attempt  to  integrate  peer  rating  research 
into  a model  of  the  processes  underlying  these  ratings,  and 
make  several  suggestions  about  promising  future  research 
d i rect ions . 


99-  Lewin,  A.  Y. , & Zwany,  A.  Peer  nominations:  A model, 

literature  critique,  and  a paradigm  for  research, (TR- 1 ) . 
Durham,  North  Carolina:  Graduate  School  of  Business 
Administration,  Duke  University,  1976. 

This  report  reviews  peer  rating  literature  and  proposes 
development  of  a decision-process  model  to  describe  the 
perceptual  processes  underlying  peer  ratings. 

100.  London,  M. , 6 Klimoski,  R.  J.  Self-esteem  and  job  complex- 
ity as  moderators  of  performance  and  satisfaction.  Journa 1 
of  Vocational  Behavior,  1975,  6,  293“ 304 . 
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Results  of  this  study  indicated  that  rater  self- 
esteem and  job  complexity  were  not  related  to  self,  super- 
visor, or  peer  ratings  of  nurse  performance. 


101.  MacKinnon,  0.  W. , Crutchfield,  R.  S.,  Barron,  F.,  Block,  J., 
Gough,  H.  G.,  6 Harris,  R.  E.  An  assessment  study  of  Air 
Force  officers  (WADC  - T R~  58  - 9 1 ( 1 ) , AD- I 51  040) . Lackland 
Air  Force  Base,  Texas:  Personnel  Laboratory,  Wright  Air 
Development  Center,  April  1958. 

Peer  nominations  of  captains  in  the  Air  Training  Command 
correlated  poorly  with  subsequent  performance.  Peer  nomina- 
tions were  only  one  of  many  assessment  variables  explored  in 
this  study. 


102.  Maslow,  A.  H.,  £ Zimmerman,  W.  College  teaching  ability, 
scholarly  activity  and  personality.  Journal  of  Educational 
Psychology,  I956,  47.,  I85-I89. 

Peer  and  student  ratings  of  college  teachers  resulted 
in  high  interrater  reliability.  Peer  ratings  loaded  highly 
on  a creativity  factor,  while  student  ratings  loaded 
primarily  on  a factor  related  to  teacher  personality. 

103.  Mayfield,  E.  C.  Management  selection:  Buddy  nominations 
revisited.  Personnel  Psychology,  1970,  23,  3 7 7 - 3 9 1 • 

Peer  ratings  of  life  insurance  sales  personnel  success- 
fully predicted  subsequent  managerial  performance,  and  this 
predictive  relationship  was  stable  across  different  groups. 

A factor  analysis  of  the  ratings  suggests  that  the  ratings 
were  based  on  a single,  general  factor  of  "overall  job 
performance."  Simplified  scoring  procedures  for  peer  rating 
scales  are  also  discussed. 


104.  Mayfield,  E.  C.  Value  of  peer  nominations  in  predicting 
life  insurance  sales  performance.  Journal  of  Applied 
Psychology.  1972  , ^6,  3 1 9~ 32 3 . 

Peer  nominations  scores  of  life  insurance  agents  at 
the  completion  of  training  successfully  predicted  sales 
effectiveness  measured  both  6 months  and  I year  after 
training.  A factor  analysis  of  the  peer  nomination  scores 
is  also  discussed. 
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105.  Mayo,  G.  D.  Peer  ratings  and  halo.  Education  and 
Psychological  Measurement.  1956,  J_6,  317-323. 

A comparison  of  Navy  airmen's  peer  ratings  of  intelli- 
gence and  effort  to  objective  measures  of  those  two 
variables  revealed  that  the  ratings  contained  halo  error. 

106.  Mays,  R.  J.  Relationships  between  length  of  acquaintance 
and  nature  of  trait  rated  and  agreement  between  raters 
(AFPTRC-TR-5V-55) • Lackland  Air  Force  Base,  Texas : Air 
Force  Personnel  and  Training  Research  Center,  November  195*4. 

This  study  shcxved  the  interrater  reliability  of 
personality  peer  ratings  by  Air  Force  OCS  candidates  to  be 
unaffected  by  length  of  acquaintance  (within  the  range  of 
3 weeks  to  5 months).  Descriptive  traits  were  rated  more 
reliably  than  evaluative  traits. 


107.  McGee,  J.  C.,  & Eaker,  R.  Clinical  supervision  and  teacher 
anxiety:  A collegial  approach  to  the  problem.  Contemporary 
Educat  ion,  1977,  j*9,  2A-28. 

This  paper  proposes  a model  for  peer  evaluation  in  team 
teaching  situations. 


108.  Michalak,  D.  A.  • Peering  at  peer  evaluation.  New  York  State 
Educat  i on , 1966,  5J5,  1 S-  1 9 - 

This  paper  suggests  procedures  for  peer  evaluation  of 
student  teachers. 


109.  Morrison,  A.,  & Hal  two  th,  H.  J.  The  perception  of  peer 
personality  by  adolescent  girls.  British  Journal  ivf 
Educational  Psychology.  1966,  3^>.  2 V I - 2 V7  . 

Adolescent  girls  rated  their  male  and  female  peers 
on  personality  traits.  The  age  and  sex  of  the  ratee  was 
shown  to  affect  the  factor  structure  of  the  ratings. 


-27* 


I 

I 


110.  Morton,  J.  B.,  6 MacBeth,  W.  A.  A.  G.  Correlations  between 
staff,  peer,  and  self-assessments  of  fourth-year  students 
in  surgery.  Medical  Education,  1977,  J_l_,  167-170. 

Medical  students  rated  their  own  performance  in  medical 
school  lower  than  did  their  peers  or  their  professors.  A 
high  correlation  was  obtained  between  peer  and  professor 
rat  i ngs . 


Mouton,  J.  S.,  Blake,  R.  R.,  6 Fruchter,  B.  The  reliability 
of  sociometric  measures.  Sociometry , 1955,  J8>,  7-^8. 

This  paper  summarizes  literature  pertaining  to  the 
reliability  of  sociometric  measures,  including  peer  ratings. 


112.  Mouton,  J.  S.,  Blake,  R.  R. , 6 Fruchter,  B.  The  validity  of 
sociometric  responses.  Soc  iomet  ry , 1955,  _[8_,  181-206. 

This  paper  summarizes  literature  that  has  examined  the 
predictive  validity  of  sociometric  measures  against  various 
performance  criteria. 

113.  Murray,  H.  G.  Predicting  student  ratings  of  college  teaching 
from  peer  ratings  of  personality  types.  Teaching  of 
Psychology , 1975,  .2,  66-69. 

Peer  ratings  of  college  teachers  on  four  personality 
traits  (leadership,  extroversion,  objectivity,  and  anxiety) 
accounted  for  most  of  the  variance  in  student  ratings  of 
teacher  performance. 


11*4.  Nassiter,  V.  , 6 Benson,  P.  Evaluating  decision  making  by 

market  managers:  A peer  rating  simulation.  Paper  presented 
at  the  meeting  of  the  American  Psychological  Association, 
Canada,  September  1978. 

This  paper  presents  rating  dimensions  for  evaluating 
the  performance  of  marketing  managers,  along  with  a summary 
of  a computer  program  (PEERRATE)  designed  to  pinpoint 
certain  biases  in  ratings  and  to  remove  them.  The  useful- 
ness of  the  dimensions  and  computer  program  was  supported  in 
part  by  results  of  a study  with  marketing  students. 


115.  Norman,  W.  T.  Validation  of  personality  tests  as  measures 

of  trait-rating  factors  (PRL-TDR-62-4,  AD-285  184).  Lackland 
Air  Force  Base,  Texas:  6570th  Personnel  Research  Laboratory, 
April  1962. 

Self-report  personality  measures  predicted  with  moderate 
success  personality  trait  peer  nominations  of  college  stu- 
dents. Also,  a factor  analysis  of  the  peer  nominations 
produced  the  same  five  factors  found  previously  by  Tupes  and 
Christal  (1958).  Predictive  validity  for  the  self-report 
measures  was  highest  when  the  surgency,  conscientiousness, 
and  culture  factors  were  used  as  criteria. 


1 1 6 . Norman,  W.  T.  Personality  measurement,  faking,  and 
detection:  An  assessment  method  for  use  in  personnel 
selection.  Journal  of  Applied  Psychology,  1963,  47, 
225-241. 

Personality  trait  peer  nominations  of  college  student 
subjects  were  used  as  criteria  for  the  validation  of  self- 
report  personality  scales. 


117*  Norman,  W.  T.  Toward  an  adequate  taxonomy  of  personality 

attributes:  Replicated  factor  structure  in  peer  nomination 

personality  ratings.  Journal  of  Abnormal  and  Social 
Psychology,  1963,  66,  574-583. 

In  reviewing  several  studies  showing  similar  factor 
structures  for  peer  ratings,  this  paper  provides  support 
for  the  stability  of  the  five-factor  structure  found  by 
Tupes  and  Christal  (1958). 


118.  Norman,  W.  T.  "To  see  ousel  as  i thers  see  us'.":  Relations 
among  self-perceptions,  peer-perceptions,  and  expected  peer- 
perceptions  of  peii  nality  attributes.  Mul t ivar iate 
Behavioral  Research,  1969.  _4,  417-443. 

Self  ratings,  expected  peer  ratings  (predictions  of  the 
ratings  of  self  made  by  peers),  and  peer  ratings  of  person- 
ality demonstrated  generally  good  convergent  and  discriminant 
validity  for  four  personality  factors  of  the  five  identified 
earlier  by  Tupes  and  Christal  (1958).  However,  agreement 
between  peer  and  self  ratings  (including  the  expected 
ratings)  was  highly  dependent  upon  degree  of  rater-ratee 
acquaintance.  The  study  also  features  a useful  comparison 
of  various  methods  for  analyzing  ratings. 
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119.  Norman,  W.  T.,  6 Goldberg,  L.  R.  Raters,  ratees,  and  random- 
ness in  personality  structure.  Journal  of  Personality  and 
Social  Psychology,  1966,  J*,  68 1 -69 1 . 1 

Results  of  a Monte  Carlo  study  suggest  that  factor 
structures  obtained  in  peer  rating  studies  may  depend 
entirely  on  the  "implicit  personality  theory"  of  the  rater. 


120.  O'Connor,  W.  F.,  t Berkshire,  J.  R.  Comparison  of  the 
pre-flight  OLQ  grade  and  the  leadership  peer  rating  aiT 
predictors  of  Training  failure  (58-I6).  Naval  Aviation 
Medical  Center,  Pensacola,  Florida:  U.S.  Naval  School  of 
Aviation  Medicine,  June  1958. 

Leadership  peer  ratings  of  Naval  Air  Cadets  predicted 
subsequent  training  failure  better  than  did  a composite  of 
military  gardes  and  leadership  ratings  by  both  peers  and 
i nst  ructors . 


121.  Pascarella,  E.  T.  Informal  interaction  and  faculty 
perceptions  of  students.  Journal  of  College  Student 
Personne I , 1975.  16,  131-lTS^ 

Self  and  peer  personality  ratings  of  college  students 
were  more  positive  (lenient)  than  the  personality  ratings 
provided  by  faculty.  Faculty  ratinqs  were  higher  for 
students  with  whom  they  had  more  informal  interaction. 


122.  Passini,  F.  T. , & Norman,  W.  T.  /\  universal  conception  of 
personality  structure.  Journal  of  Personality  and  Social 
Psychology . 1966,  4_, 

The  authors  gathered  self  and  peer  personality  ratings 
of  college  students  who  had  known  each  other  for  only  15 
minutes.  The  self  and  peer  ratings  showed  very  little 
correspondence.  A factor  analysis  of  the  peer  ratings 
revealed  the  same  five  factors  previously  reported  by  Tupes 
and  Christal  (1958).  The  authors  speculate  that  persons  may 
enter  a rating  situation  with  a common  "implicit  personality 
theory"  on  which  they  base  their  ratings. 
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123-  Passini,  F.  T. , 6 Norman,  W.  T.  Ratee  relevance  in  peer 
nominations.  Journal  of  Applied  Psychology,  1969,  53, 
185-187. 

This  paper  discusses  the  use  of  a statistic  for 
estimating  the  degree  to  which  peer  nominations  reflect 
characteristics  of  the  ratee.  The  authors  conclude  that 
this  index  of  ratee  relevance  has  potential  use  in  the 
evaluation  of  peer  nomination  scales. 

124.  Pease,  0.  Comparing  faculty  and  school  supervisor  ratings 
for  education  students.  College  Student  Journal,  1975, 

9.  91-94. 

Instructor,  supervisor,  peer,  and  self  ratings  of 
education  students  i ntercor re  1 a ted  highly.  Also,  global 
peer  ratings  of  performance  showed  good  interrater  reli- 
ability. 


125.  PepinsLy,  P.  N.  The  meaning  of  "validity"  and  "reliability" 

as  applied  to  sociometric  tests.  Educational  and 
Psychological  Measurement,  1949,  39*^9 • 

The  terms  "validity"  and  "reliability"  are  defined  in  a 
sociometric  context,  with  discussion  applicable  to  peer 
rat  i ngs . 

126.  Powell,  M.  G.  Comparisons  of  self  rating,  peer  ratings,  and 
expert's  ratings  of  personality  adjustment.  Educational  and 
Psychological  Measurement . 1948,  8,  225-234. 

Little  agreement  was  found  between  self,  peer,  and 
counselor  ratings  of  ma lad justment  symptoms  in  college  coeds. 

127-  Prien,  E.  P. , & Leo,  R.  J.  Peer  ratings  and  leaderless  group 
discussion  for  evaluation  of  classroom  performance. 
Psychological  Reports.  1965,  Ij6,  59-64. 


-31- 


Peer  ratings  of  group  behavior  in  a classroom  setting 
showed  low  interrater  reliability  over  several  weeks,  while 
test-retest  reliability  was  higher.  A factor  analysis 
of  the  peer  ratings  revealed  the  following  factors:  overall 
effectiveness  (general  factor),  interpersonal  effectiveness, 
and  productivity.  The  "interpersonal  relations"  scale  scores 
of  the  peer  ratings  correlated  highly  with  class  test  grades. 


128.  Prien,  E.  P. , £ Woodley,  K.  Note  on  reliability  of  peer 
ratings  of  classroom  performance.  Psychological  Reports, 
1971.' 28 , 89-90. 

Results  of  this  study  indicated  that  the  interrater 
reliability  of  peer  ratings  increased  with  the  amount  of  time 
group  members  were  exposed  to  each  other. 

129.  Ramey,  V.  W.  The  relationship  of  peer  group  rating  to 
certain  individual  perceptions  of  personality.  Journal  of 
Experimental  Education,  1958,  2_7,  1^3“  1^9- 

Graduate  students  rated  themselves,  members  of  their 
immediate  peer  group,  and  persons  with  whom  they  were 
less  well  acquainted  on  items  selected  from  the  California 
Psycholog ica I Inventory.  Persons  rated  themselves  more 
like  members  of  their  peer  group  than  like  members  of  the 
"more  distant"  group. 
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130.  Reynolds,  H.  H.  Efficiency  of  sociometric  ratings  in 
predicting  leadership  success.  Psychological  Reports.  1966, 

19.  35-40. 

Air  Force  ROTC  student  peer  ratings  of  leadership 
ability  showed  good  validity  in  predicting  subsequent 
leadership  performance  as  measured  bv  staff  officer  ratings. 

! , 
1 J 

131.  Ricciuti,  H,  N.  Ratings  of  leadership  potential  at  the 
U.S.  Naval  Academy  and  subsequent  officer  performance. 

Journal  of  Applied  Psychology.  1955,  39,  1 94- 1 99 - 

Positive  but  low  correlations  were  obtained  between 
peer  ratings  of  Naval  Academy  students'  "apt i tude-for- 
service"  and  post  graduate  on-the-job  performance. 
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132.  Ricciuti,  H.  N.,  £ French,  J.  W.  Analysis  of  ratings  of 
leadership  potential  at  the  U.S.  Naval  Academy.  Amer i can 
Psycholog i st , 1951,  6^  392. 

Peer  and  supervisory  ratinqs  of  leadership  potential  of 
Naval  Academy  cadets  were  found  to  be  suitable  criteria  for 
test  validation  purposes. 


133.  Rigby,  M.  K. , Hoffman,  E.  L. , Rohrer,  J.  H.,  £ Wilkins,  W. 

Three  approaches  to  peer  evaluation.  American  Psychologist, 
1953,  8,  421  - 

Using  Marine  Corps  enlisted  men  as  subjects,  three 
approaches  to  peer  evaluations  were  compared:  rankings; 
sociometric  ques t i onna i res ; and  behav iora 1 1 y-anchored  peer 
rating  scales.  The  three  methods  correlated  highly  with 
each  other  and  with  staff  ratings. 


134.  Roadman,  H.  E.  An  industrial  use  of  peer  ratings.  Journa 1 
of  Applied  Psychology,  1 9^4 , 48.  211-214. 

Peer  ratings  of  middle-level  managers  successfully 
predicted  subsequent  promotions,  and  participating  managers 
viewed  the  peer  rating  experience  as  educational, 
constructive,  and  non- threaten i ng . 

135-  Rokeach,  M. , Gladin,  L.,  £ Trumbo,  D.  A.  Two  validation 
studies  with  high  and  low  dogmatic  groups.  In  M.  Rokeach 
(Ed.),  The  open  and  closed  mind.  New  York:  Basic  Books, 
I960,  p.p.  105-124. 

These  studies  found  that  peer  ratinqs  provided  a better 
assessment  of  student  charac ter i st ics  than  did  ratings  by 
faculty  members,  presumably  because  students  had  more 
opportunity  to  observe  their  peers  in  unstructured 
s i tuat  ions . 


-33- 


136.  Ronan,  W.  W.  , Anderson,  C.  L. , 6 Talbert,  T.  L.  A psycho- 
metric approach  to  job  performance:  Firefighters.  Publ i c 
Personnel  Management,  1976,  5_,  t»09-1422. 

Peer  and  supervisory  ratings  of  firefighters  did  not 
correlate  well  with  each  other  or  with  objective  performance 
tests,  though  the  ratings  did  show  good  within-group  inter- 
rater reliability.  Because  of  low  correlations  between 
ratings  and  performance  tests,  the  authors  conclude  that 
peer  and  supervisory  ratings  are  useful  only  for  tapping 
dimensions  not  assessed  by  performance  tests,  such  as 
motivational  and  interpersonal  aspects  of  job  performance. 

137-  Roodin,  P.  A.  Birth  order  and  cooperativeness:  Peer 
ratings.  Psychological  Reports,  1971,  29,  590. 

College  student  peer  ratings  of  cooperativeness 
correlated  near  zero  with  the  birth  order  of  the  ratee. 


138.  Rossman,  J.  E.  Teaching,  publication,  and  rewards  at  a 
liberal  arts  college.  Improving  College  and  University 


Teaching,  1976,  2A_,  238- 2^0. 


In  this  study,  peer  ratings  of  college  instructors  were 
not  related  to  academic  rank  and  correlated  only  minimally 
with  salary,  while  students'  ratings  were  not  related  to 
either  rank  or  salary.  Peer  and  student  ratings  did 
correlate  highly. 


139.  Rubinstein,  E.  A.  Analysis  of  self  and  peer  personality 
ratings  of  psychotherapists  and  comparison  with  patient 
ratings.  Journal  of  Consulting  Psychology,  1958,  22^,  10. 

This  paper  presents  evidence  suggesting  that  peer 
ratings  of  personality  traits  by  psychotherapists  are  based 
on  more  general  factors  than  are  their  ratings. 


1A0.  Saal , F.  E. , & Landy,  F.  J.  The  mixed  standard  rating 

scale:  An  evaluation.  Organ i zat iona I Behavior  and  Human 
Performance , 1977,  13,  1 9 ~ 3 5 - 


Peer  and  supervisory  ratings  of  police  officer  job  per- 
formance were  gathered,  using  both  behaviorally  anchored 
scales  and  mixed  standard  scales.  Ratings  obtained  from 
the  mixed  standard  scales  showed  less  halo  and  leniency 
error,  but  also  lower  interrater  reliability. 


141.  Schmidt,  F.  L.  , 6 Johnson,  R.  H.  Effect  of  race  on  peer 
ratings  in  an  industrial  situation.  Journal  of  Appl i ed 
Psychology , 1973,  57,  237-241  . 

Peer  ratings  were  gathered  from  foreman  trainees  who 
had  undergone  human  relations  training.  The  peer  ratings 
showed  no  racial  bias  and  also  possessed  good  convergent 
and  discriminant  validity,  thus  suggesting  that  peer  ratings 
may  be  useful  in  racially  integrated  groups. 

142.  Schmitt,  N. , & Hill,  T.  E.  Sex  and  race  composition  of 

assessment  center  groups  as  a determinant  of  peer  and 
assessor  ratings.  Journal  of  Applied  Psychology,  1977,  62, 
261-264.  — 

This  study  investigated  racial  and  sex  bias  in  peer 
ratings  of  managerial  candidates  at  an  assessment  center. 
Results  suggested  a small  degree  of  bias,  dependent  on  the 
racial  and  sex  composition  of  the  assessee  group. 

143.  Schumacher,  C.  F.  A factor-analyt ic  study  of  various 
criteria  of  medical  student  accomplishment.  Journal  of 
Medical  Education,  1964,  39.,  192-196. 

Several  measures  of  medical  school  performance  were 
intercorrelated  and  factor  analyzed,  including  grades. 
National  Board  Examination  test  scores,  and  peer  ratings. 
Almost  all  criteria  loaded  highly  on  a "general  medical 
knowledge"  factor,  but  peer  ratings  loaded  on  a factor 
termed  "skill  in  relationships  with  patients." 


144.  Simpson,  R.  H.  Rating  patterns  for  maximizing  competition 

and  minimizing  number  of  comparative  judgments  necessary  for 
each  rater.  Journal  of  Applied  Psychology,  1953,  37, 
290-292.  ' ' 
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This  paper  presents  patterns  for  assigning  ratees  to 
raters  for  use  in  situations  where  a large  number  of 
individuals  in  a group  prevents  all  persons  from  rating 
al I others . 


1^5.  Smith,  G.  M.  Usefulness  of  peer  ratings  of  personality  in 
educational  research.  Educational  and  Psychological 
Measurement , 1967,  2_7,  9^7'98A . 

Peer  ratings  of  college  students  on  personality  traits, 
especially  ratings  of  strength  of  character,  were  shown  to 
have  both  good  interrater  reliability  and  good  predictive 
validity  for  academic  success.  Also,  the  peer  nomination 
technique  is  discussed,  and  a factor  analysis  of  the  A2 
personality  variables  rated  is  presented. 

1A6.  Springer,  0.  Ratings  of  candidates  for  promotion  by 

co-workers  and  supervisors.  Journal  of  Applied  Psycholoav, 
1953,  37,  3^7-351 • 

Peer  and  supervisory  ratings  of  the  promotab i 1 i ty  of 
industrial  workers  showed  a low,  positive  correlation  with 
subsequent  promotions.  Interrater  agreement  between  super- 
visors was  much  higher  than  between  peers.  Supervisory 
ratings  tended  to  be  lower  than  peer  ratings,  and  both 
groups  provided  lower  ratings  on  global  criteria  than  on  the 
more  specific  dimensions. 

1A7-  Stahl,  M.  J.  Innovation  and  productivity  in  research  and 

development:  Some  associated  individual  and  organ  i zat  iona 1 

variables  (AF 1 T-TR-7fe~ 1 O) • Wr ight-Patterson  Air  Force  Base, 
Ohio:  Air  Force  Institute  of  Technology,  1976. 

Peer  ratings  of  innovation  and  productivity  by  scien- 
tists and  engineers  were  used  as  criteria  for  assessing  the 
effects  of  organizational  variables  on  performance  in 
research  and  development  laboratories. 


148.  Stefflre,  B.  , King,  P. , £ Leafgren,  F.  Characteristics  of 
counselors  judged  effective  by  their  peers.  Journal  of 
Counseling  Psychology,  1962,  % 335“ 340 . 

Highly  reliable  peer  ratings  of  counseling  effectiveness 
were  obtained  from  counselors  using  a Q-sort  technique. 

Those  persons  rated  most  favorably  by  their  peers  were  shown 
to  have  higher  academic  performance,  vocational  interests 
more  congruent  with  their  position,  and  lower  scores  on  a 
dogmatism  scale  than  those  rated  lower. 

149.  Suci,  G.  J.,  Val lance,  T.  R. , & Glickman,  A.  S.  A study  of 
the  effects  of  "1 ikingness"  and  level  of  objectivity  on  peer 
rating  reliabilities.  Educational  and  Psychological 
Measurement , 1955,  J_6,  1 47*  1 52 . 

Neither  administering  a popularity  questionnaire  before 
gathering  the  peer  ratings  nor  varying  the  objectivity  of 
the  rating  scales  had  a significant  effect  on  the  reliability 
of  Naval  OCS  candidate  peer  ratings. 


150.  Swanson,  R.  G. , £ Johnson,  D.  A.  Relation  between  peer 
perception  of  leader  behavior  and  instructor-pilot  per- 
formance. Journal  of  Applied  Psychology,  1975»  60,  198-200. 

Peer  ratings  of  Air  Force  instructor  pilots  on  the 
Consideration  and  Structure  leadership  dimensions  correlated 
significantly  with  a composite  performance  criterion. 

151.  Titus,  H.  E.  The  use  of  peer  nominations  as  a predictor  of 
academic  success  in  college.  Journal  of  Experimental  Edu- 
cation, 1969,  37.'  83-66. 

High  interrater  reliabilities  were  obtained  for  peer 
ratings  of  college  students'  academic  potential,  and  the 
ratings  successfully  predicted  grade  point  average. 
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1 52 . Trites,  E.  K.  Adaptability  measures  as  predictors  of  per- 
formance ratlnqs.  Journal  of  Applied  Psychology,  i960,  M . 
349-3S3- 

Peer  ratings  of  Air  Force  pilot  trainees,  in  combination 
with  supervisory  ratings  and  expert  ratings  by  psychologists 
of  motivation,  aptitude,  and  personality  trails,  successfully 
predicted  subsequent  performance  as  an  officer. 

153.  Tucker,  M.  F.,  Cline,  V.  B. , S Schmitt,  J.  R.  Prediction  of 
creativity  and  other  performance  measures  from  biographical 
information  among  pharmaceut i ca 1 scientists.  Journal  of 
Applied  Psychology,  1 967 , 5J_,  1 3 1 - I 33 . 

This  study  found  that  peers  and  supervisors  based  their 
creativity  ratings  of  pharmaceutical  scientists  on  different 
factors . 


lb!*.  Tuddenham,  R.  D.  Studies  in  reputation:  I.  Sex  and  grade 
differences  In  school  children's  evaluations  of  t he i t peers. 
II.  The  diagnosis  of  social  adjustment.  Psycho  log icai 
Monographs , 1952,  6b  ( I ) . 

School  children's  peer  ratings  of  reputation  success- 
fully predicted  subsequent  social  adjustment. 

155.  Tupes,  E.  C.  Relationships  between  ratings  by  peers  and 
later  pe r for m.ince  oTTlSAF  Off! C e r f~UKn  Jute  5cl)ooT  gradu- 
ate «'a (AF*>TRC~-TN-57“12fi»  AD-134  257).  Lackland  Air  Force 

Base,  Texas:  Air  Force  Personnel  and  Training  Research 
Center,  October  1967. 

A composite  of  peer  ratings  of  personality  traits  by 
Air  Force  OCS  candidates  correlated  very  highly  with 
subsequent  Officer  Effectiveness  Reports.  The  peer  ratings 
showed  higher  validity  than  two  other  predictors,  military 
and  academic  grades,  but  predictive  validity  was  highest 
when  all  three  of  these  predictors  were  used  together. 


156.  Tupes,  E.  C.  Personality  trait  related  to  effectiveness  of 
junior  and  senior  Air  Force  officers  (WADC-TN-59-198. 

231  256).  Lackland  Air  Force  Base,  Texas:  Personnel  Lab- 
oratory, Wright  Air  Development  Center,  November  1 959 - 

This  study  extends  the  findings  of  Tupes  and  Christal 
(1958)  to  senior  officers.  Results  indicated  a common  factor 
structure  underlying  the  personality  trait  peer  ratings  of 
junior  and  senior  Air  Force  officers.  The  peer  ratings  of 
the  senior  officers  successfully  predicted  subsequent  per- 
formance, as  was  the  case  for  junior  officers. 

157*  Tupes,  E.  C.t  Borg,  W.  R. , 6 Friedman,  G.  A factor  analysis 
of  the  OCS  paired  comparison  evaluation  system  (Technical 
"Report  53-10).  Lackland  Air  Force  Base,  Texas:  Human 
Resources  Research  Center,  Air  Research  and  Development 
Command,  June  1953. 

This  report  describes  a factor  analysis  of  paired  com- 
parison peer  evaluations  made  by  Officer  Candidate  School 
students.  High  correlations  between  dimensions  led  to  a 
large  general  factor  and  the  authors  therefore  suggested 
using  the  paired  comparison  technique  only  for  an  evaluation 
of  overall  performance. 


153.  Tupes,  E.  C.,  6 Christal,  R.  E.  Stability  of  personality 
trait  rating  factor  obtained  under  diverse  conditions 
(WAD'C-'TN- 53-61 , AD-151  0^1).  Lackland  Air  Force  Base, 

Texas:  Personnel  Laboratory,  Wright  Air  Development  Center, 
May  1958. 

A five  factor  structure  (surgency,  agreeableness,  de- 
pendability, emotional  stability,  and  culture)  underlying 
personality  peer  ratings  was  found  to  be  invariant  across 
different  samples,  raters,  situations,  and  rater-ratee 
acquaintance  time. 


159.  Tupes,  E.  C.,  £ Christal,  R.  E.  Recurrent  personality 

factors  based  on  trait  ratings  (ASD-TR-61 -97 . AD-2&7  778) . 
Lackland  Air  Force  Base,  Texas:  Personnel  Laboratory, 
Aeronautical  Systems  Division,  May  1 96 1 . 
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Personality  trait  ratings  from  several  studies  yielded 
the  same  five  factors:  surgency,  agrceableness , depend- 
ability, emotional  stability,  and  culture. 


160.  Tupes,  E.  C. , & Kaplan,  M.  N.  Similarity  of  factors  under- 
lying peer  ratings  of  socially  acceptable,  socially  unaccept- 
able 7 and  bipolar  personality  traits  (ASP-TN-6l -^8 , 
OTTT  Lackland  Air  Force  Base,  Texas:  Personnel 
Laboratory,  Aeronautical  Systems  Division,  June  1 96 1 . 

Peer  ratings  of  students  at  the  Air  Force  Command  and 
Staff  School  were  factor  analyzed  and  the  resulting  factors 
were  very  similar  to  the  Tupes  and  Christal  (1953)  factors. 


161.  Tupes,  E.  C.,  £ Kaplan,  M.  N.  Relationships  between  per- 
sonality traits,  physical  proficiency,  and  cadet  effective- 
ness reports  of  Air  Force  Academy  cadets  (aSD-TN-61-53, 

AD* 2(jA  916)  • Lackland  Air  Force  Base,  Texas:  Personnel 
Laboratory,  Aeronautical  Systems  Division,  September  1961. 

Personality  trait  peer  ratings  of  Air  Force  cadets, 
officer  candidates,  and  officers  all  correlated  highly  with 
performance.  Factor  analysis  of  the  peer  ratings  revealed 
underlying  factors  similar  to  those  identified  previously 
by  Tupes  and  Christal  (1958)  with  a sixth  factor,  physical 
ability,  emerging  probably  because  rating  scales  related  to 
physical  skills  were  added  to  those  employed  by  Tupes  and 
Chrl stal . 


162.  Turk,  H.  Instrumental  and  expressive  ratings  reconsidered. 
Sociometry , 1961,  2j±,  76-81. 

This  study  showed  that  student  nurse  raters  identifying 
highly  with  a task  tended  to  rate  peers  who  also  identified 
highly  with  the  task  as  more  physically  attractive  than 
peers  who  identified  less  highly  with  the  task. 

163.  Walder,  L.  0.,  Abelson,  R.  P. , Eron,  L.  D.  , Banta,  T.  J.,  £ 
Laulicht,  J.  H.  Development  of  a peer  rating  measure  of 
aggression.  Psychological  Reports,  1961  , 9^  A97~556. 
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This  paper  discusses  the  development  of  a peer  rating 
scale  to  measure  aggression  in  grade-school  children.  The 
scale  showed  a moderate  degree  of  Interrater  reliability. 

164.  Waters,  L.  K. , 6 Waters,  C.  W.  Peer  nominations  as  pre- 
dictors of  short-term  sales  performance.  Journal  of 
Applied  Psychology,  1970,  54_,  42-44. 

Peer  ratings  of  sales  trainees  successfully  predicted 
subsequent  sales  performance  and  peer  rating-performance 
relationships  were  not  affected  by  friendships  among  the 
trainees. 


165.  Webb,  W.  B.  Self-evaluation  compared  with  group  evaluations. 
Journal  of  Consulting  Psychology,  1952,  J_6,  305“ 307. 

Peer  and  self  ratings  of  personality  traits  by  college 
students  correlated  minimally.  Individuals  tended  to 
evaluate  themselves  especially  positively  on  traits  the 
group  valued  highly. 

166.  Webb , W.  B.  The  problem  of  obtaining  negative  nominations 
in  peer  ratings.  Personnel  Psychology,  1955,  8_,  61-64. 

This  paper  suggests  that  positive  nominations  provide 
most  of  the  predictive  power  in  peer  evaluations,  and 
argues  that  in  many  cases  negative  nominations  need  not  be 
sol ici ted. 

167.  Weitz,  J.  Selecting  supervisors  with  peer  nominations. 
Personnel  Psychology,  1958,  1 I , 25-36. 

Peer  ratings  o'  the  supervisory  potential  of  life 
insurance  agt  .ts  ^jccessfully  predicted  subsequent  per- 
formance as  a supervisor. 

163.  Wherry,  R.  J.,  C Fryer,  0.  H.  Buddy  ratings:  Popularity 
contest  or  leadership  criteria?  Personnel  Psychology, 

1949,  2,  147-159- 
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Peer  ratings  of  the  leadership  potential  of  Army  OCS 
candidates  were  shown  to  be  better  predictors  of  subsequent 
leadership  performance  than  ratings  by  instructors.  The 
study  also  showed  peer  ratings  to  be  highly  reliable. 


I69.  Wherry,  R.  J.,  Stander,  N.  E.,  6 Hopkins,  J.  J.  Behavior 
trait  ratings  by  peers  and  references  (WADC-TR-59"360 , 

AD-239  098)7  Lackland  Air  Force  Base,  Texas:  Personnel 
Laboratory,  Wright  Air  Development  Center,  December  1 959 • 

This  study  showed  that  ratings  of  OCS  appl icants  by 
persons  listed  as  references  on  their  applications  possessed 
poor  interrater  reliability  and  validity.  The  authors  sug- 
gest that  peer  ratings  are  a more  useful  assessment  approach 
than  are  references. 


170.  Wicas,  E.  A. , & Mahan,  T.  W. , Jr.  Characteristics  of 
counselors  rated  effective  by  supervisors  and  peers. 

Counselor  Education  and  Supervision,  1966.  £.  50-56. 

Counselors  rated  as  highly  competent  by  their  peers 
scored  more  favorably  on  personality  self-report  measures 
than  did  counselors  rated  lower  in  competence. 

171  Wiggins,  N.,  Blackburn,  M. , 6 Hackman,  J.  R.  Prediction  of 
first-year  graduate  success  in  psychology:  Peer  ratings. 
Journal  of  Educational  Research,  1969.  63.  81-85. 

~ ' v 

Peer  ratings  of  personality  traits  and  intellectual 
qualities  by  beginning  graduate  students  successfully 
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